The Annals of Applied Probability 

2006, Vol. 16, No. 4, 1816-1850 

DOI: 10.1214/105051606000000376 

© Institute of Mathematical Statistics, 2006 



PERIODICITY IN THE TRANSIENT REGIME OF 
EXHAUSTIVE POLLING SYSTEMS 

By I. M. MacPhee, M. V. Menshikov, S. Popov^ and S. Volkov 
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and University of Bristol 

We consider an exhaustive polling system with three nodes in its 
transient regime under a switching rule of generalized greedy type. 
We show that, for the system with Poisson arrivals and service times 
with finite second moment, the sequence of nodes visited by the server 
is eventually periodic almost surely. To do this, we construct a dy- 
namical system, the triangle process, which we show has eventually 
periodic trajectories for almost all sets of parameters and in this 
case we show that the stochastic trajectories follow the deterministic 
ones a.s. We also show there are infinitely many sets of parameters 
where the triangle process has aperiodic trajectories and in such cases 
trajectories of the stochastic model are aperiodic with positive prob- 
ability. 

1. Introduction. A polling system has nodes where jobs arrive and 
queue and a single server which switches between the nodes to process the 
jobs. In exhaustive polling systems the server processes all jobs at its current 
node i, say, including any that arrive while jobs there are being processed, 
before switching to another node j, chosen by some rule. Conditions for 
transience/recurrence of polling systems using a greedy switching rule were 
given in [5, 6]. There is a critical case which has been investigated in [8] 
and [9]. In this paper we show that, for an exhaustive polling system with 
N = 3 nodes, with arrival streams and service times putting the system in 
its transient regime and switching according to a threshold rule (a type of 
generalized greedy rule), the sequence in which the server visits the nodes 
is eventually periodic for almost all choices of threshold parameters. 
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To show this, we consider in Section 2 the embedded Markov chain in Zl^ 
where we observe the pohing system at service and switching time comple- 
tions. We construct a dynamical system using the vector field of expected 
drifts of the chain for the server at each node. As the polling system is tran- 
sient, this dynamical system exits any finite ball eventually, so we project it 
onto the unit simplex. With = 3 nodes, we call this the triangle process, 
as it lives in a triangle — it has piecewise linear trajectories that change di- 
rection when they meet the triangle boundary so it looks something like a 
billiards model, but where "refiections" are caused by changes of dynamics 
due to the server switching to another node. We call any trajectory which 
returns to its start point after finitely many server switchings an orbit. 

It is worth mentioning that this deterministic system closely resembles 
(and in some cases is) the so-called affine interval exchange transforma- 
tion model. However, the affine interval exchange transformation here is 
contracting so it is not a bijection and it reverses orientation so it is not 
order-preserving. It is thus rather different from the usual interval exchange 
transformation model that has been much studied during the past 30 years. 
We comment more on this in Section 5. 

We state our main results in Section 3. Theorem 3.1 states that for a.s. 
all choices of switching thresholds the triangle process has a finite number 
of orbits and every trajectory converges toward one of these orbits. We say 
the triangle process is stable in this case. That there are no more than four 
orbits for any triangle process parameters is the content of Theorem 3.2. 
In Theorem 3.3 we show that there are (infinitely many) choices of the 
switching thresholds that lead to the existence of aperiodic trajectories of 
the triangle process. These results are proved in Sections 4 and 5. Using 
these results for the deterministic triangle process, we show, in Theorem 3.4, 
that the projections of the trajectories of the stochastic queueing model a.s. 
converge onto the orbits of the triangle process, when it is stable. This 
implies the periodicity of the sequence of nodes receiving service for the 
polling system. Finally, in Theorem 3.5, we show that when the triangle 
process has nonperiodic trajectories then, in some situations, the sequence 
of nodes visited by the stochastic model is periodic with positive probability 
and aperiodic with positive probability. These results are proved in Sections 
6 and 7. 

2. System description. An exhaustive polling system has A^ nodes where 
jobs queue and a single server which switches to the next node j, chosen 
with some rule, after processing all jobs at the current node i, including 
any that arrive while jobs at i are being processed. In the general model 
switching takes a time which depends upon the pair i, j. We will assume 
the following at each node i: the arrival processes are independent Poisson 
streams with arrival rate Aj; the service times are i.i.d. and independent of 
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the arrivals with finite mean /x^^ and variance af < oo; the switching times 
are independent with finite means that depend upon the initial and final 
node while the server just waits for the next arrival at any node when it 
completes service at i and finds the system is empty. 

For the results in our paper, we will assume that the switching times are 
zero. We believe our argument can be extended to a more general situation, 
but our main interest is in the transient case, where the system behavior 
is not sensitive to the switching time distributions, though of course it is 
affected by the switching rule. 

The methods we know for showing that periodicity of the node sequence 
for the deterministic triangle process implies the same for the stochastic 
process are applicable only to Markov processes. As the sequence of nodes 
visited is determined by the jump chain, we will consider the discrete time 
process 

^ = mt);s{t))}, t = o,i,..., 

where ^(t) describes the queue lengths and s{t) the server location at the 
epochs of service time and switching time completions. Its state space is 
X {1, 2, ... , N}. That H is well defined, irreducible and aperiodic follows 
from the assumptions of Poisson arrivals and finite first moments of the 
service times. For later convergence arguments, we also require finite second 
moments of the service times and under these conditions, standard results 
imply this embedded chain is essentially equivalent to the continuous time 
process as regards transience/recurrence. The transition probabilities can 
be computed via the Laplace transforms of the service times, but we do not 
need them explicitly at any point. A standard conditioning argument readily 
produces the expected one-step mean drifts 

miit + 1) - mmty,s{t)) = {x-j)) = x^^if - 

(1) 

i = l,...,iV, 

for X G with Xj > 1 and the server at node j and we make considerable 
use of these. A similar result holds for expected drifts during switching 
times, but we will not detail these as, in fact, we will assume switching to 
occur instantaneously. Other arrival and service processes for which there 
is an embedded Markov chain can be found, but their treatment needs no 
significant extension of our methods. 

We will start by stating the known explicit conditions for recurrence/tran- 
sience. For exhaustive polling models, the conditions for recurrence/transience 
depend only upon the total loading (or traffic intensity) 

N 

p = ^ Pi where pi = \i/ pi at each node i 
1=1 
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for a great many (and seemingly all sensible) switching rules under the 
assumption that switching times have finite first moment. We have the fol- 
lowing: 

Theorem 2.1. The process H is positive recurrent if p<l, transient if 
p>l. 

Remarks. Foss and Last [5] establish the result for a more general 
model. The method of proof is via Lyapunov functions as described in [1, 4] 
or [10]. The papers by MacPhee and Menshikov [8] and Menshikov and 
Zuyev [9] consider the critical case /O = 1, where the behavior of the system 
depends strongly on the first two moments of the switching time distribution. 
Foss and Last [6] obtain the result of this theorem for nonexhaustive polling 
systems under a greedy switching policy. 

Generalized greedy switching rules. When the server has completed all 
the tasks at its current node, it chooses its next node using a switching 
rule which we will assume depends upon the queue lengths, that is, the rule 
is a function m : dR'^ ^ {1, 2, . . . , iV}, where dR^ = [jj{y G : yj = 0}. 
Any y{ must satisfy 7^ j if yj = and a variety of such rules have 

been studied in the literature. We will study only a class of generalized 
greedy rules defined as follows. For each node j, there is a vector of positive 
weights bj = {bji, . . . ,bjN) and at states y E dR^ with yj = 0, = i, 

where bjiyi = maxk{bjkyk) (for our results it is not important how ties are 
resolved). The simple greedy rule is the special case where all bjk are equal. 

A deterministic model. Our subsequent analysis of the transient case 
is based around the following deterministic model of the system. Consider 
a particle moving in x {1,...,A^} with linear dynamics given by the 
one-step mean drifts of H as calculated in (1). With its position denoted 
y = (2/I) 2/2, • • • , Un) and the server at node j, we see that at (y, j) with yj > 
the particle has velocity 

i=l \ i^j I 

where the Cj denote the axial unit vectors in M^. If a point with yj = is 
reached, then a different set of dynamics (corresponding to the server switch- 
ing to another node) is chosen instantaneously according to some generalized 
greedy switching rule 9^. 

The trajectory y(t) of our particle is constructed as follows. From start 
point yito) = y G RI^ with y^ > and the server at node j, the particle 
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travels along the line 



through y with the appropriate velocity. If pj > 1, the particle never reaches 



while if pj <1, the particle reaches 



(2) y(ti) = E(y. + ^^)e.G«^^ 



Pj - Xj 



at time t = tQ + Vj/O- ~ Pj) —'■ Now the server switches to node 

and the next and subsequent pieces of the trajectory are computed as above. 



The node process. In the transient case the trajectory exits from any 
finite ball eventually, so, in order to study whether there is any periodicity 
in the order the server visits the queues, we project onto the unit simplex. 
Project the lines (■j{y) onto the hyperplane 5i = {y G : X]?, Ui = 1} using 

A:M^\{0}^cSi where A(y) = -^, 

which maps each line ij{y) onto the intersection of Si with the plane con- 
taining ij{y) and the origin. The feasible positions for the particle will be 
mapped onto points in the unit simplex = {y G R^ ■ J2i 2/^ = !}• start- 
ing points y and ay for any a > 0, the lines ij{y) and ij{ay) have exactly 
the same image in Si, so we can restrict our attention to reference points 
y G dSi , the boundary of S^ . 

Under the condition pj > 1, the server in the stochastic process can remain 
serving at queue j indefinitely, so we will only consider the cases where pj < 1 
at all queues j. Under the condition pj < 1, we see from equation (2) that 
the trajectory y{t) leaving y G dSi along £j{y) next reaches dM.^ in finite 
time at the point 

Y] (m + ) ei with projection V — + ^Vj ^. ^ 

f^.\ Pj-XjJ f^. {pj - Xj) + pjOjyj 

where = {y e ■Y^iyi = 1, Vj = 0, ?/i > for z / j} foi j = 1,2, . . . , N are 
regions of the switching boundary dS^ and 

(3) ej:=/i7^EAi-l, j = l,2,...,N. 



There is a very nice geometric description of the projected process. For 
any fixed j, the lines ij{y) through different points y are parallel so they 
are concurrent after projection, that is, their image lines on Si are either 
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Fig. 1. The node process when N = 3. 



parallel or share a common focus, vj , say. The line £j (y) with y = J2i^j ^i^i + 
(Aj — fJ'j)ej passes through so Vj is the point where £j{y) meets Si if this 
happens. This is the case whenever 6j ^0, in which case 



When 9j = 0, the ij{y) with y G 5i all lie in Si and the projected lines 
through different y are parallel. 

The focus point Vj will be outside Si when 6j < or when 6j > and 
Pj = Xj/fij < 1, conditions which correspond to the next switching event 
taking place after some finite time. 

As our aim is to study the switching sequence, we assume from now on 
that 

N 

(5) /Cj < 1, i=l,...,N and p = ^pj>l 

i=l 

(which means that the whole system is transient, yet the server does not 
get stuck at any individual node). We can now define the discrete dynam- 
ical system Z = {z{t)},t = 0, 1, . . . , living on = = dS^ . Define 
mappings 

f^.{i^j-Xj) + tijejZj J ^ 
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The image under fj of a point z £ lies at the intersection of the hne 
through z and vj and the boundary region as shown for the case = 3 
in Figure 1. For given z(0) G dS^' let 

z{t + l) = ip{z{t)), i = 0,l,..., 

N 

where (p{z) = '^I{vi{z)=j}fj{z),ze A°. 
i=i 

We will call Z the node process, as it records information about the projec- 
tion of the dynamical system y{t) only at switching epochs. 

3. The three node case. From this point our analysis is restricted to the 
case where the system has three nodes and the switching decision is made 
using a generalized greedy rule. This enables us to give clear statements of 
our results and methods, but is complex enough to be very interesting in 
our opinion. 

Recall that when the server has completed all jobs at queue i, the switch- 
ing rule is defined by parameters bij > and selects the next node k when 
bijXj < bii^Xk for i,j,k any permutation of 1, 2, 3. Projection by A onto 
S]_ reduces the switching boundaries {x:bijXj = hkXk, Xi = 0} to decision 
points, one on each side of the triangle A^ as shown in Figure 2. This rule 
can be applied to the stochastic process H and the dynamical process y{t) 
as stated with the following resolution of boundary cases. For the stochastic 
process S, a randomized rule may be used whenever bijXj = bikXk, so we will 
consider the consequences for the node process Z of both possible decisions 
at such points. Specifically, we will consider trajectories of the node process 
which branch when they exactly hit the decision points. From now on, we 
will call the node process Z the triangle process and specialize our notation. 
The construction in Section 2 is somewhat abstract, but the system which 
we have defined and wish to study is really very simple to describe. Figure 2 
shows a trajectory starting from z(0) = z near 63. This is mapped to z{l) 
along the line from z to ^3 and then to z{2) just below di. As z{2) is toward 
62 from di, the next point z(3) is on the line from z{2) to V2- From here 
the trajectory continues toward ^3, then vi, then back toward and from 
there toward V2 and so on. 

We note that the mapping which determines the trajectories is not contin- 
uous at the di. However, during extensive numerical investigation, we found 
that, for all the configurations of rates and decision points we tried, the 
trajectories we examined converged toward periodic orbits. 

Triangle process notation. Let k denote the values 1, 2, 3 or either of 
its cyclic permutations 2, 3, 1 or 3, 1, 2 and let i,j, k denote any permutation 
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Fig. 2. The triangle process trajectory. 



of 1, 2, 3. For z = {I — x)ej + xe^ G A? with x G [0, 1], we define Side(z) := i, 
7r(z) := X and write z = {x,i) (with standard right-hand axes, increasing x 
corresponds to going around the triangle anticlockwise). 

For the triangle process Z, any generalized greedy rule 9\ has the form 

for values = (1 + 6j^/6jj)~^ G (0, 1) for i=l,2, 3. The decision points {di,i) 
will usually be written simply dj. We will usually refer to rules of this type 
as threshold rules. 

For z = {x,i),z' = {x',j), we will use distance \z — z'\i := \\z — z'\\/\/^, 
the Euclidean distance scaled so that when i = j, \z — z'\i = \tt{z) — tt{z')\ = 
\x — x'\, that is, it is length along the side of the triangle. 

The forward mapping if defined in (7) is 1-1 except at the di which have 
two images, while the inverse of ip is 1-1 where it exists. Call z a pre-image 
of z' when z' = ip^^\z) for some t > 1. 

Definition. A sequence = 0, 1, . . . , satisfying (7) is a trajectory 

of Z. If z(0) is a pre-image of a decision point dj, then there are at least two 
trajectories starting from ^(0). 

We will study closely the sets = ip{A^~^).,t = 1,2, ... , which are such 
that A* contains the possible locations for z{t) from all initial points z(0) G 
A9 . Alternatively, ^ \ A* is the set of points with fewer than t pre-images, 
a description which we will use later. The set = '^{AP) is a strict subset 
of A^ for any decision points since, for example, ei ^ A^ ,i = 1,2,3. Hence, 
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Fig. 3. The triangle process. 



D D ■ ■ ■ D for i = 2, 3, Further, A^ is a union of three disjoint 

closed intervals, one in each A^. We introduce now the notation Aj = A^CiA^ 
for later use — A\ is depicted in Figure 3. As only intervals containing a 
decision point will be split by ip, it follows that A* is a union of at most 3t 
disjoint closed intervals. 

Periodicity definitions. We call a trajectory z(t), t = 0, 1, . . . , eventually- 
m-periodic if there is > such that, for all n> N, 

Side(z(n + m)) = Side(z(n)). 

A trajectory which is not eventually-m-periodic for any m is called nonpe- 
riodic. 

We say that Z has a periodic orbit if there is a finite sequence of points 
ui,U2, ■ ■ ■ , Um with (j){ui) = Uj+i for i = 1, 2, . . . , m — 1 and (pi'^m) = ui- The 
sequence Side(ui), Side(ti2), • • • , Side(iim) is the node-cycle of the orbit. We 
will consider cyclic permutations of orbits/node-cycles to be equivalent to 
the original orbit /node-cycle. All orbits are disjoint since, for any point u 
on an orbit, (u) is uniquely defined for all t > so there is no point at 
which two orbits could join. 

A trajectory converges onto an orbit ui, . . . ,Um when there exists no > 
such that 

\z{mt + n + no) — — > as i ^ oo for each n = 1,2, . . . ,m, 

that is, for each n, the subsequence of the trajectory corresponding to phase 
n of the node-cycle converges to Un- For any point z = {x,i) G A^ and < 
£ <x, let 

(8) Me{z) = {z' E 4° : |7r(z') - x\i < e} 
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denote the e-neighborhood of n, J\f^{z) = {z' G vl^ : < 'k{z') — x < e] and 
M- (z) = {z' G ^0 : < X - 7r(z') < e} the one-sided ^-neighborhoods of z. 
We will say an orbit is stable when, for each Un, there exists e > such that 
trajectories starting from any z G Me{un) converge onto the orbit. When 
trajectories started from z £M^{un) but not from J\f~{un) (or vice versa) 
converge onto the orbit, we say the orbit is stable on one side and otherwise 
we say the orbit is unstable. An orbit containing a decision point must be 
unstable if it is of odd length m and may be stable on one side if m is even. 

We are now in position to state our main results for the triangle pro- 
cess Z. These hold for any set of parameters Aj, /ij, i = 1,2,3, satisfying 
conditions (5). 

Theorem 3.1. For almost all decision points di G AP,i = 1,2,3, the tri- 
angle process Z has finitely many periodic orbits. For such sets of decision 
points, all trajectories z[t) are eventually periodic and each converges onto 
one of these orbits as t ^ oo. 

The quantifier "almost all" is used in the sense of Lebesgue measure x 
counting measure on [0, 1] x {1, 2, 3}. Due to the convergence behavior of all 
trajectories, we will call this the stable case. Typically, in this case all the 
orbits are stable, but unstable orbits are possible when an orbit includes a 
decision point. 

Theorem 3.2. For any set of decision points di,i = 1,2,3, there are at 
most four periodic orbits. 

The next result shows that not all choices of decision points result in all 
trajectories being eventually periodic. 

Theorem 3.3. There is an uncountable set of decision points for which 
Z has nonperiodic trajectories. 

These results have important consequences for the behavior of the under- 
lying stochastic process H = We now introduce the random times 
T( , t = 1, 2, . . . , at which the server changes queues. Under the assumptions 
in (5) on the parameters, the Tt are all a.s. finite. The next result concerns 
the stochastic triangle process 

(9) C(t)=A(e(rt)), t = l,2,.... 

This process has trajectories living on A^, so we can use the same definition 
of convergence for it as for the triangle process. 
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Theorem 3.4. Suppose the service times have variances af < oo. For 
any set of decision points (^1,^2,^3) such that Z is stable, the stochastic 
process ( is also stable in the sense that a.s. each trajectory of Q converges 
onto one of the periodic orbits of Z . 

Theorem 3.5 (No zero-one law). There exist configurations of the de- 
cision points such that the stochastic process C, has nonperiodic trajectories 
with positive probability. 

We construct an example with trajectories that can converge to a periodic 
orbit of Z with positive probability and also can be nonperiodic with positive 
probability. 

Some open problems. 

• It seems from numerical computations that there are, in fact, at most 
three orbits. Is this correct or are there examples with four orbits? 

• Is it also true that, for any given set of decision points di, the sets of 
parameters X,fi where there are nonperiodic orbits has measure zero? 

• Which of the results proved here also hold when there are four or more 
nodes? 

• Can the stochastic process C converge with positive probability to an orbit 
of even length which contains a decision point and is only stable on one 
side? 

4. Proofs for the triangle process. The conditions for transience or re- 
currence of the stochastic process S are in terms of the traffic intensities 
Pi in accord with the intuition that considering weighted work at nodes 
rather than just the numbers of queued jobs should not affect such prop- 
erties. Such a re- weighting also helps simplify the treatment of the triangle 
process. To describe its effect, we introduce the fractional linear functions 
Fa : [0, 1] — > [0, 1], where, for any a > 0, Fa{x) = x/{a + {1 — a)x). These 
have a key composition property F^o Fp = Fa/3 ■ 

Lemma 4.1 (Re- weighting). Consider the triangle process Z with param- 
eters Xi, di for i = 1, 2, 3 and define T:A^^A^ by T{z) = F^^i^.{z),z G 

j4?,z = 1,2,3. Z is isomorphic to the triangle process Z' with parameters 
p[ = 1, \[ = Xi/ Pi = Pi and d[ = T{di), i = l,2, 3. Specifically, for any given 
z{0) G A^, if Z' is started from T{z{0)), then z'{t) = T{z{t)) for t = 1,2, ... . 

Proof. We start by assuming that only parameters Aj, pi for some 
i are transformed by multiplication by a > and briefly describe an iso- 
morphism between trajectories of the dynamical processes y and y'. Define 
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Tj :M.^ by Ti{y) = y + {a — l)yiei, so Fj rescales W\_ by a in the 

direction. Let ij{y) and denote the hnes through y parallel to the tra- 
jectories of y{t) and y'{t) respectively when the server is at node j. Then 
Ti{£j(y)) = £^ (rj(y)). This space rescaling provides an isomorphism between 
entire trajectories whenever the switching decisions are identical, or, equiv- 
alently, r'(rj(y)) = r(y) for all y € dR\. 

The class of threshold policies is closed under space rescalings Tj with 
the parameters bji being mapped to abji and to abj^~ with consequent 
changes to dj and df_^. Now for any y £ dM.^ and z = A(y) = {x,i), we find 
that z' = A{Ti{z)) satisfies 

({x,i), i = i, 

z' = I (.Fi/aix),i), i=j, 
[{Fa{x),i), i = k, 

where {i,j,k) is a cyclic permutation of (1,2,3). Applying successive rescal- 
ings by 1/ fJLi in the direction for i = 1, 2, 3 and using the composition 
property of F„ gives the result. Note that Fa satisfies Fq(0) = 0, Fa{l) = 1 
and Fa{x) > 0, x G (0, 1), so the mapping T fixes the corners Cj of and 
smoothly rescales the sides. □ 

The implication of Lemma 4.1 is that we need only study the triangle 
process with parameters fii = fi2 = fJ-3 = I and general Aj and di to under- 
stand the possible behavior of the general case. We will work with the rates 
fJ-k = ^ from this point. We now find, from (3), that 6k = J2n Pn — ^ ='■ & > 
for each k as we are in the transient case. It follows from standard projective 
geometry results that the foci vj. are the vertices of an equilateral triangle 
V with sides parallel to the A^, as shown in Figure 3. It can also be seen 
directly as, from equation (4) for each k, 

1 1 

Vk = -^{Piei + PjCj + {pk-l)ek) and, hence, Vi - Vj = -{ej - Ci). 

Under conditions (5), the Vk are outside , as remarked after equation (4). 
Let ^A denote those points of ^4^^ which are internal to V and ^Ai = A^Ci ^A. 
In Figure 3, ^A contains all of A^ except the neighborhood of ei. 

When pk < or, equivalently, pi + pj > 1, the line from Vi to Vj meets 
A^ at the point (1 — a)ej + aeu with a = pk/6 £ (Oi !)> while if pi + pj < 1, 
then ek £ ^A. If ^ ^A, let J-^ = {z £ A9 : x > p-JO} U{z£A°y.x<l- p-JO} 
denote the corner of ^4'' containing and lying outside ^A. Let J^ = when 
ej;,£^A (so J2 = J3 = 0, but Ji / in Figure 3). 

The next two lemmas state key properties of the node process mappings 
but ignore the effect of the decision points. Translating definition (6) into 
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the triangle process notation, we find that fj-.A^UA^^ A^ satisfies 



fj{x, i) = (- ^'^1 r , and 

\l- Pj + 6{1- x) J 

(10) 



1 — pj + 6x 

and /j(ej) = /j(0, i) = fc) = Pi/{pi + Pj^), which demonstrates continuity 
of the fj at the corners ej of A^. 



Lemma 4.2 (Contraction property). The function fj is monotone on 
A9 U ^4? and there exists 7 G (0,1) and Cj{^) C A9 U A? such that for z, 

z' G Cj(7) n A^, n = i, k, we have \ fj{z) — fj{z')\i < j\z — z'\\ and further, 



Cji-y) D {{x,i):0< x < min(l, p^/^)} U {(x. A:) : max(0, 1 - {pi/9)) < x < 1}, 
so Cj{'y) D ^Ai U ^Aj^. The containment is strict i/ J? U / 0. 



Proof. On A9, let g{x) = 7r(/j(x,i)) = pi{l - x)/(l - pj + 6^(1 - x)). 
As g'{x) = —pi{l — Pj)/{Pi + Pfc ~ ^x)"^ < 0, (7 is monotone decreasing in x. 
Choose 75 such that if > 0, then |g'(l)| = Pj/(1 — pj) < 7i < 1, while if 
p^ < 6, \g'{pf^/9)\ = (1 — P])/ Pi < 7i < 1- The same argument applies for 
selecting on ^9 using g{x) = (pjx + (1 — pj) (1 — x))/(l — pj + 0x) . Finally, 
choose 7 > max(7j,7^) and set Cj(7) = {z G j4? U ^4? : |g'(x)| < 7}. □ 

The mapping /j may not be contracting near ej or if they are not in 
^A. This implies that (p may not be contracting around, say, (x, 1) if ps < 6, 
di is close to 63 and p-^/6 < x < di. 

Starting from z ^ A^, we can apply either fj or to z, then again apply 
either /j or fk to /} (^) G and so on. Let a G {1,2,3}^ denote a sequence 
with the properties ai / i, at ^ o"t_i, i = 1, 2, ... (such a sequence will be 
called allowed sequence). For z G A^, let fa\z) = fatifat-A' " " ifaiiz) • • •)). 
As the mappings fi are monotone, they are invertible and we will need, a 
little later, to consider mappings fa ^\z) = fi^ ^\fat-i{' ' " (/o-i ^\z) ■ ■ ■)). 



Lemma 4.3. There exists a constant k > such that, for any short 
enough interval [u,w] C ^Ai, i G {1,2,3}, and any allowed sequence a, with 
the constant 7 G (0, 1) from Lemma 4.2: 

(i) |/W(u;) - /(*)(n)|, < 7V - nil /or t = 1,2, ... , 
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\w — u\i 

for any v G (u, w) . 

Proof. Inequalities (i) follow immediately from Lemma 4.2 since we 
have fj(^Ai) C ^Aj for all pairs ij^j. For part (ii), consider any monotone 
function g on a short interval [a,b] with |(7'(x)| > ai > and < a2- 

Expanding g to second order around a with Taylor's theorem, we find that, 
for any c G (a, b), 

g{c)-g{a) c-a 



9{b)-g{a) b- 



-{\ + rf) where rj = 0(b — a) 



and is nonzero by monotonicity of g. Composing this result t times, we 
obtain 



lfi'\w)-f!,"{u)h I""-"!! 



n=l 



where \r]n\ < ki\w — n|i7"' for some ki > independent of u, v, w and a. 
This establishes the second set of inequalities. Intuitively, the idea here is 
that lines from points v G [u, w] to vertex vj will be almost parallel when 
|w; — n|i is small, so relative lengths of subintervals will be about the same 
after mapping by fj. □ 

From Lemmas 4.2 and 4.3, it follows that there is < 7 < 1 such that 
mapping 99 is uniformly contracting on (^(7) := Ci^j) U Cj(7) U C^(7)- Fix 
this 7 from now on. 

Lemma 4.4. All points z on a periodic orbit lie in ^A. 

Proof. We show instead that points outside ^A, that is, z ^ J^^ for some 

k, cannot lie on a periodic orbit because they have terminating sequences of 
pre-images. Suppose 7^ 0. 

If neither or dj G J^, then ^{Jj^) C A^H^A and no z G has a pre- 
image. If dj G Jj, but di ^ Jy then, recalling the notation 7r(z) = x for z = 
{x,i), those z £ A^ n Jj^ with 7r(z) < 7r(/j(dj)) have a single pre-image, while 
the other z £ Jf, have none. The case with G but dj ^ is similar. 

If both dj, dj G Jf,, as shown in Figure 4, then fj{di), /j(dj) G J^, but no 
z e{z e A9: 7t{z) > 7r(/i(dj))} U {z G A? : tt{z) < 7r(/j(dj))} has a pre-image 

under 93. Let it = and observe that fa '^^\z) — > as t — > 00 



TRANSIENT POLLING SYSTEMS 



15 




for any z G A? n Jj^. Hence, for z with tt{z) > 7r(/j(dj)), we see that 

(~2t) 

■n{fa {z)) < 7r(/j((ij)) for some finite t and these z have only finitely many 
pre-images under 93. Any point in n can be written as fj{z) for some 
z G A^ n Jj^, so they too have finitely many pre-images in this case. We note 
that a forward trajectory from any z £ Jj^ either enters ^A or converges onto 
the period two orbit on the points {pf./d,i) and (1 — p^jQ^f). □ 

Lemma 4.5. For any starting point z{0), there is to > such that z{t) G 
C(7) for allt>to. 

Proof. We see that ^A is closed under if, so once a trajectory enters 
it never leaves. Any trajectory that never enters ^A must remain in one of 
the Jj and so, by Lemma 4.4, converges to the two cycle on the endpoints 
of that Jj. In either case Lemma 4.2 implies that it enters the contracting 
region and remains there. □ 

Lemma 4.6. For any eventually-m-periodic trajectory z{t), t = 0, 1, 2, . . . : 

(i) there exists an orbit ui, . . . , Um of period m onto which the trajectory 
z{t) converges; 

(ii) trajectories z'{t) with the same node-cycle converge onto the same 
orbit; 

(iii) there can be at most one orbit having a given node-cycle. 

Proof, (i) Note that, as z{t) is eventually-m-peiiodic, there exists to > 
such that Side(z(t + m)) = Side(z(t)) for t > to- Now Lemma 4.5 implies 
there will be ti > such that z{t) G C{'j) for all t > ti. Since Side{z{tm -\- i)) 
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is the same for all t > to, Lemma 4.2 implies 

\z{{t + l)m + i) - z{tm + i)\^ < 7™|z(tm + i) - z{{t - l)m + i)\-^ 

and, hence, Ui = liuit^oo z{tm + i) exists. For parts (ii) and (iii). Lemmas 
4.4 and 4.2 imply that, for some r <m, \z{t) — z'{t + r)\i — > exponentially 
quickly as t — > oo, so the trajectory z'{t) converges onto the orbit ui, . . . , 
Urn ■ The existence of another orbit vi, . . . , Vm with the same node-cycle is 
impossible because we can choose z'{0) =vi. □ 

Recall the notation ^*+^ = ip{A^) = ■■■ = The set A* contains 

all points that have at least t pre-images under (p. Let V = {(/?^~*^ (dj) :i = 
1, 2,3;t = 0, 1,2,.. .} be the set of pre-images of the decision points and V 
be its closure. Let \z — V\i = inf^/g-p \z — z'\i denote the distance from z to 
V — of course, \z — V\i = \z — V\i. 

Remarks. If {di,d2,ds} n = for some finite t, then V is finite. If 
{di, d2, d^} f] ^ for any finite t, then V can still be finite when there is 
a finite orbit containing one or more of the decision points. The only other 
possibility is that at least one decision point has infinitely many distinct 
pre-images and so V is infinite. An orbit containing a decision point must 
be unstable if it is of odd length m and may be stable on one side if m is 
even. 

Lemma 4.7. (i) IfV is finite, then the triangle process has a finite num- 
ber of periodic orbits and each trajectory z{t), from any starting point z{Q), 
converges onto one of these orbits as t^ oo. 

(ii) Whether V is finite or not, if z ^ V and the trajectory z{t) with 
z{0) = z converges onto the orbit ui, . . . , Um, then, for some e > 0, there is 
a unique trajectory starting from each z' £M£{z) and it converges onto this 
orbit. 

(iii) Any orbit ui, . . . , Um with Un^V for each n = l, . . . , m is stable. 

Proof, (i) As V is finite, we can partition ^4'^ into \ V\ open intervals 0„ 
(indexed anticlockwise from the interval containing ei , say) with points of V 
as endpoints. These 0„ are never split by mapping with (p as they and their 
images never contain decision points, so each 0.„ is mapped into another 
by if as it is continuous and monotone except at the di. Thus, (p induces 
a mapping h of the indices {1, 2, ...,1^1} into itself and, hence, h has a 
core -fC C {1, 2, . . . , 1^1} such that h{K) = K. As h has no fixed points, 
it permutes K and so factors into a product of disjoint nontrivial cycles 
and, hence, all trajectories starting from z are periodic with one of a 
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finite number of node-cycles. Eacli node-cycle supports only a single orbit 
by Lemma 4.6 and these trajectories converge onto one of these orbits. 

Now consider trajectories with z{0) = z gV. If, for any t > 1, we have 
z(t) G On for some n, then the trajectory is periodic and converges onto 
a finite orbit by the above argument. This only leaves trajectories with 
z(t) S V for all t>0. As was remarked when orbits were defined, all orbits 
are disjoint, so a trajectory can only remain in V by following a single orbit. 

(ii) As z ^V, the trajectory started from z never hits a decision point and 
so never branches. As V is closed, there exists e > such that \z — V\i > e, so 
no trajectory started from any z' €Afs{z) ever branches. As M£{z) contains 
no pre-images of decision points, the same switching decisions are made for 
each of these trajectories at all times t, so they have the same node-cycle 
and by Lemma 4.6, they converge to the orbit ui, . . . , Um- Part (iii) follows 
immediately by considering z{0) =ui. □ 

We are now ready to establish Theorems 3.1 and 3.2. Let /i denote 
Lebesgue measure on in the following proofs. This measure is consis- 
tent with the distance | • |i we are using and any null sets in ^4'' will remain 
so under change of the parameters, as the re- weighting in Lemma 4.1 is 
smooth. 

Proof of Theorem 3.1. It suffices to show that almost all choices 
of decision points di lie outside the ^* they generate after some finite t. It 
follows from Lemmas 4.4 and 4.2 that fJ-{A^) — > as i ^ oo for any decision 
points di , but this is not sufficient since lim inf ^ is a function of the di . 
What we show is that, for any given parameters pi, there is a measure zero 
set of locations for the decision points where they could have infinitely many 
pre-images. 

We start with the case Ci^^A or, equivalently, Jij^0, i = 1,2,3. Let 
Ji = /^( J.) and J*+^ = fi{Jj U Jl) for i = 1, 2, 3 and t = 1, 2, . . . (here {j, k} = 
{1,2,3} \ {i}). These sets are well defined as C ^Ai and, hence, J* C ^Ai 
for all t. Further, JiriJl = and, hence, J*+^ n Un=i J? = ^ ^^i all t. Let 
/* = VAi \ Ui Jf and note that 1*+^ = /^(/j U 4). 

We initially consider only ^Ai. At stage t' , the set IJ^ J" consists of 2*'+^ — 
1 disjoint intervals interleaved with the 2* "'"^ intervals forming /* . The set 
J* "^^ contains 2* "^^ intervals, each interior to one of those of /* . Recall 
that is the measure induced by the distance | • |i. Let K > he such that 
//(//) < j = 1, 2, 3. As Ei=iKJr) < K^Ai) < 1, it follows that 

/x( J*) — > as t — > oo. 

From Lemma 4.3(i), each subinterval of /j , j = 1, 2, 3, has length bounded 
by 7*' for some 7 < 1. Now from Lemma 4.3(ii), it follows, by summing over 



18 



MACPHEE, MENSHIKOV, POPOV AND VOLKOV 



all the subintervals in /*' and mapping sequences, that 

< e^^''K/i(J*'+*+^), t = 2,3, . . . 

and, hence, /u(/*) — > as t — > oo. The set /* contains all points in that 
could have t or more pre-images, given appropriately chosen decision points. 
Hence, If° = Hi^ -^j- contains those points that could have infinitely many 
pre-images and /Li(/f°) = . 

This construction is independent of the choice of the decision points and 
works on all three sides of ^A. With = IJ^ we have shown IJ,{I°°) = 0, 
that is, ^-almost all choices of the decision points have only finitely many 
pre-images under the mapping ip that they define. The theorem now follows 
in this case from Lemma 4.7. 

It remains to modify this argument in the case where one or more of the 
Cj € ^A. For any such corner, Jj = 0, so let Jj'(e) = {|z — ei|i < e} when € 
^A, jf = Ji otherwise, where e > is small. The previous double mapping 
process will produce overlapping sets, so we modify it by setting j} = /{{Jf) \ 
(J° U 4) and then J*+^ = fi{jj U 4) \ (J° U J^) for i = 1, 2, 3 and t = 
1, 2, .... This change ensures the previous disjointness properties and the 
construction goes through without further change, except that now some of 
the sub-intervals of /* may be empty. As before, we conclude that = 0, 

though here does depend upon e. 

To relate this construction to the existence of pre-images, let P„ = {{di,d2, 
ds) :all z S J^{l/n),i = 1,2,3, have no legitimate pre-images under ip}. Then 
[Jn'^n = A'(xA^xA^. For any fixed n, = P„ n has /i(P^) = and 
hence fi{{jfV^) < J2T = 0. Now the result follows as before. □ 

Proof of Theorem 3.2. We consider first the case where P = (di) : 

i = 1, 2, 3; t = 0, 1, 2, . . .}, the set of pre-images of the decision points, is finite. 
A point z € P is of type i when it is a pre- image of dj. V splits A'^ into closed 
intervals M„, n=l, 2, which are never split by iterated mappings 

with if. Hence, each M„ contains points from at most one periodic orbit and 
as If is contracting on ^A, each M„ can contain only a single point from an 
orbit — in cases where the shared endpoint of M„ and M^+i lies on an orbit, 
that is, this orbit contains one or more decision points, there can be at most 
one other orbit with a point in Af„ U M^+i- 

The number of orbits is limited by each di being at the boundary of just 
two intervals. Consider an orbit with a point u G Mt which has an endpoint 
ip^~'''\di). The point Lp^^'\u) lies on the same orbit but occupies one of the 
M„ neighboring dj, or equals d^. By the preceding paragraph, there are at 
most six orbits. 

We can reduce the bound as there are at least two intervals M„ with 
endpoints of different type, ij and ik, say. Suppose the first is Mt with 
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endpoints <p^~^\di) and ip^~^\dj) with r < s. The interval ip^^\Mt) has 
endpoints and (p^^~'^\dj) so it neighbors dj. (^^*-*(Mj) sits inside an interval 
neighboring dj. If Mt contains a point from an orbit, then this orbit occupies 
two of the six intervals neighboring the decision points. If Mt contains no 
point from any orbit, then neither can one of the intervals neighboring dj. 
Either way, the maximum number of orbits is now only five. Repeat the 
argument with the interval with endpoints ik and there are at most four 
orbits in the stable case, that is, where V is finite. 

We now extend this argument to the case where V is infinite — let V' = 
{z €V : z has infinitely many pre-images}. Suppose there are some finite or- 
bits and consider any of them. It cannot include any points in V' , so choose 
e > small enough that \un — di\i > e for every ti„ on the orbit and each 
di^V' . It follows that — Lp'^~'''\di)\i > e for every Un on the orbit and 
every point in V' for suppose this was not true. Select the ip^~^\di) with 
minimal r within e of n„. By Lemma 4.3 (i), it follows that 

\(p^''\un) - dill ^ - (p''~''\di)\-^ < e, 

in contradiction to our choice of e. Now our argument for the stable case 
applies with the added possibility that one or more di may be limit points 
(from one or both sides) of V further restricting the opportunities for orbits. 
□ 

We show later that it is possible for finite orbits to exist simultaneously 
with nonperiodic trajectories in this nonstable case. In our study of this 
model we simulated the triangle process over the whole range of its pa- 
rameters Pi and dj. To avoid problems with rounding, we used an algebraic 
representation of the process which is described below in Section 5. The 
representation there of the decision points has to be finite in practice, so the 
results of the computer analysis were sometimes inconclusive, for example, 
the implemented algorithm sometimes produced trajectories that were not 
trapped by an orbit in the number of steps we could accurately calculate. 
Otherwise, the algorithm identified either 1, 2 or 3 orbits and never more. 
Therefore, we conjecture that Theorem 3.1 can be strengthened, by re- 
placing at most four orbits by three orbits. We did not manage to prove this 
analytically, though. That three orbits can exist is easily seen after a little 
numerical work with Figure 5 as a guide. 

5. The triangle process in the nonstable case. We now show that there 
are locations for the decision points where at least one of them has infinitely 
many pre-images and nonperiodic trajectories exist. This argument necessar- 
ily uses yet another way of describing the triangle process. We will consider 
only the case where each Jj = 0, but our construction can be carried out in 
other cases too, with some limitations. 
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Fig. 5. An example with three orbits. 



We first construct a binary representation z = i: xiX2X^ . . . with each xt € 
{0, 1} for all points of A^. For each z G A?, z = 1, 2, 3, set [recall the notation 
7r(-) from Section 3] 

rO, if7r(z)<7r(/5(e,-)), 
^1 ll, if7r(z)>7r(/5(e5)). 

To continue this construction, set = {/j(ej)} and Bi~^^ = fi{Bj U BjJ for 
1 = 1, 2, 3 and t=l, 2, .... 

Each set B- contains 2*~^ points which, by monotonicity of the mappings 
fi, interleave the 1 + 2*~^ points of Ur=i and, hence, split in two each of 
the 2*~^ intervals created up to stage t — 1. Label the br & Bju -B- clockwise 

and the Ur G Ur=i anticlockwise so that = ej and 

7r(a,._i) < 7r{fi{br)) < 7r(a,.), r = 1,2, . . . , 2*"^ 
and for z £ [ar-i,ar), set 

rO, if7r(z)<7r(/5(6,,)), 
"^*-ll, if^(z)>7r(/j(6,,)). 

This encodes every point of with a unique binary code up to the usual 
indeterminacy for points terminating with infinite strings of zeros or ones, 
for example, ei = 2 : 111 • • • = 3 : 000 • • • . 

This binary encoding can be used to define a distance between points 
z = i: xiX2 ■ ■ • ,z' = i: x'^x'2 • • • by 

00 

(11) \z-z'\^ = Y^\xt-x\\-2-' 

t=i 
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and we note the following: (i) ordering of points on by tt{z) corresponds 
to lexicographic ordering on the binary sequence x; (ii) any sequence of 
points which is convergent under | • |i remains convergent under \ ■ \h and 
vice versa; (iii) this metric induces a measure fi), on A^. 

In defining the required decision points, we will make much use of the 
inverse mapping ip, where 

for i = 1,2,3. When ip^~^\z) exists, then we will say ip is legitimate and we 
have iIj{z) The mapping ^ can be described quite simply using 

the binary encoding. For z = i: X1X2 ■ ■ ■ with xi = 0, then ipiz) = k: 2/12/2 • • ■ j 
where yi = 1 — X2, 2/2 = 1 — and so on. In general, let w = 1 — w for 
w € {0, 1}, X = X1X2 ■ ■ ■ and with this notation ip maps 

(12) i:Ox — >k:x and i:lx — >j:x 

for 1 = 1, 2, 3. 

The forward mapping if thus can be represented as 

i:x — >j:Ox if X < d,- and i:x — >k:lx if x > dj. 

Now represent each point of A^ by a point in the interval [0, 1] using the 
mapping 



I : X\X2 ■ ■ 



1 1 °° 



and ignore, for the moment, the branching at the decision points. The for- 
ward mapping 93 becomes 



(13) ^(x) 



|x + l, for X G [di,(i2) U [(is, 1), 



2 

-^x + |, for X G [0,(ii) U [^2,(^3), 



for X G [0, 1), where d\, , G [0, 1) are the points corresponding to the 
decision points. The mapping above can be regarded as a particular case 
of the affine interval exchange transformation model (see, e.g., [2, 7]). The 
mapping defined by (13) is contracting and is not order-preserving, which 
makes it different from the usual and well-studied case of interval exchange 
transformations (see, e.g., [3, 11] and references therein). Note also that, for 
interval exchange transformations with contracting and/or flips, it is natural 
that cycles do exist (see [3]), as happens in our model. 
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5.1. A decision point with infinitely many pre-images. We are now ready 
to construct decision points providing a nonstable case. We will use q = 1001 
and r = 0110, two four digit sequences such that q = r and such that 
successively maps z = i:qx and z = i:rx as follows: 

z = i: lOOlx — > j : llOx — >k:Qlx — > j : Ox — >i:x = V'^^^ {z) 

and, similarly, 

z = £ : OllOx — >k: OOlx — > j:lQx — >k:lx — >i:x = ^j^^'^ {z). 
The decision points have the form 

di = l: qrx • • • , 
(14) d2 = 2:1010100000---, 

ds = 3 : 0100000 • • • , 

where x, the infinite tail of the sequence for di, will be constructed re- 
cursively at a later point to ensure that di has infinitely many legitimate 
V'-images. As the notation di identifies the triangle side, we will sometimes 
use it to denote just the binary sequence. Note that, under lexicographic 
ordering, r = 0110 < 1001 = q. 

For di to have infinitely many legitimate ^'-images, it must certainly have 
one and the intervals with at least one legitimate ip-ima.ge are 

on Al [Ods, ld2\ = [0101111 • • • , 1010101111 • • •], 

on Al [0di,ld3] = [Orgx 110111111 •• •], 

on Al [0^2, Idi] = [00101011111 • • • , Irqx • • •], 

which the reader can readily check certainly contain di and the ijj^^\di) for 
t = 1, 2, . . . , 8. We will construct the rest of the binary sequence for di so 
that its V'-iniages always fall in these three intervals by finding a sequence 
of quadruples q and r that guarantees legitimacy. 

Extended legitimacy property. The sequence di = 2/12/22/3 • • • where 2/1 = 
1-, y2 = f and 2/t G {q-.i'} t = 3,4, ... has extended legitimacy if, under 
lexicographical ordering, 

(a) when 2/t = r, then yt+iyt+2 ■■■>di, 

(b) when 2/t = q, then yt+iyt+2 ■■■<di, 

where g > r as remarked above. 

At each t > 3 this property ensures the legitimacy of ^Z'*-'**"^^ (di) for r = 3, 
2, 1, 0. We will now construct an aperiodic sequence with extended legit- 
imacy and show that this means there are decision points leading to the 
existence of aperiodic orbits which is crucial for establishing Theorem 3.3. 



TRANSIENT POLLING SYSTEMS 



23 



A 




Fig. 6. Staircase diagram for sequence qrqqrqrq.... 



Fix an irrational a G (1, 2) and some (3 G [—1, 1]. Let y = y{a, (3) = 2/12/22/3 ' ' ' 
be a sequence of quadruples q and r with yi = q, y2 = r and for t = 3,4, . . . , 



where Qt = Qt{y) = Z)r=i l{y^=q} and Rt = t — Qt, that is, Qt is the number 
of times q appears in the sequence yi?/2 • • - yt- This sequence is defined using 
successive rational approximations to the irrational number a as shown in 
Figure 6, which shows a staircase approximating the line y = ax + (3. 

Lemma 5.1. There are uncountably many sets of decision points where 
at least one of them has infinitely many pre-images. 

Proof. We first show that, for any irrational a > 0, the sequence y' = 
y{a,(3') is lexicographically less than y" = y{a,(3") whenever —l<(3'<f3"< 
1. 

Suppose y' and y" are not equal. Then there is t > 1 such that y!^ = y!^ 



for n<t, but y^+i / yf+i. Evidently, Qt{y') = Qt{y") and Rt{y') = Rt{y"), 
so the only way for y^+i / v't+i is 



yt+i = 




\il + Qt<a{l + Rt)+f5, 



otherwise. 



a{l + Rt{y'))+[3' <l + Qt{y') 
= l + Qt{y") 
<a{l + Rt{y"))+(3' 



ill 
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in which case y'^^i = r < q = y"^i- As the fractional parts of ma, m = 1,2, . . . , 
are dense in (0, 1), the inequahties (3' <n — ma < (3" are satisfied for in- 
finitely many pairs of integers m, n and so the above inequalities also show 
that y' = y" is not possible. 

Now consider the sequence y = y{a, 0) which starts qr ■ ■ ■ . Each r is fol- 
lowed by a g and there cannot be more than two qs in succession as a < 2. 
Suppose yt = r. Then the sequence yt+iyt+2yt+3 ■ ■ ■ coincides with y{a,(5t), 
where 

(3t = aRt-Qt>0 

and so yt-i-iyt+2yt+3 • • • > y. If instead yt = q, then either y^+i = r so that im- 
mediately we have yt+iyt+2yt+3 • • • < y or yt+i = q, in which case yt+iyt+2yt+3 
coincides with y{a,l3t), where 

= aRt-Qt<0 

and so yt+iyt+2yt+3 •••<?/• In all cases we see that the sequence y has 
extended legitimacy at t for all t > 3. As a is irrational, aRt — = is not 
possible and, hence, there is no t such that yt+iyt+2yt+3 • ■ • = y from which 
it follows immediately that y is aperiodic. 

To establish this lemma, let di = l: y{a, 0) and d2, be as defined in (14) 
for some irrational a € (1,2). By construction, di has an infinite sequence 
of legitimate -(/'-images or, equivalently, an infinite sequence of pre-images 
under if. □ 

Remarks. For the above set of the decision points, there is also a period 
3 orbit: (a, c, 6), with the points of the orbit given by 

1:101010---10---, 
(15) 3:101010---10---, 

2:101010---10---. 

We mention here that originally we were not sure that the nonstable case 
was possible and unsuccessfully tried to prove this using the triangle process 
description of this section. The specific combination of decision points pro- 
ducing the nonperiodic orbits described above was discovered by studying 
computer output (of a programme written to generate sample trajectories) 
to assess various lines of investigation. 

5.2. The nonstable case has fib-fneasure 0. The measure on AP in- 
duced by the metric | • |;, is generally singular with respect to Lebesgue 
measure, but the set of decision points with infinitely many ip pre-images 
has yUfo-measure 0. 
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Suppose we select the decision points uniformly, that is, each digit in the 
binary sequence is or 1 with chance 1/2 independently of the other digits. 
Then fif^ a.s. we can write, for some k>0, 

k times 

This means the interval in with legitimate ^/^-images has left endpoint 

00__^1---. 

fc+l times 

The Borel-Cantelli lemma tells us that fib a.s. there is a subsequence in the 
binary sequence for di of the form 

y = 00 — 00 10 00 — 00 10 00 — 00, 

m times m times rn times 

where ?n > A; + 3 is even. Now consider the sequence V^W(di), t > 1. For 
some r, starts with either the above sequence y or with y and for 

some s < 2m + 6, ip^^'^'''^ (di) is a point on with binary sequence starting 
with at least k + 2 zeros and so its next ip-im&ge is not legitimate. It follows 
immediately that the set of decision points di with infinitely many pre- 
images has /Xfo-measure 0. 

5.3. Properties of the deterministic system when V is infinite. For the 
triple of the decision points constructed in (14), di has infinitely many pre- 
images and since ^"^(^2) = 1 : 000 • • • = 62 and ^'"^(f^s) = 1 '■ HI ' " " = 63, 
both d2 and d^ have finitely many pre-images under ip, unlike di. Through- 
out this section, the triple of the decision points is assumed to be such that 
di has infinitely many pre-images, while d2 and d^ have only finitely many. 
We also assume in this subsection that ^A = A^ so that ip is contracting 
everywhere. 

Let Vi = {dl,'^p{dl),^lJ^'^\dl), . . .} be the infinite set of pre-images of di un- 
der if which, by assumption, are all legitimate. We will study the properties 
of this set. Let Vi denote the closure of the set Vi- 

Lemma 5.2. Each u£Vi is a limiting (accumulation) point ofVi. 

Proof. It is sufficient to show that di is a limiting point for Vi. Indeed, 
since the mapping doubles the distance | • 1^ between points, if di is limiting 
for a sequence ui, U2, ■ ■ ■ C Vi, then 'ip^^\di) is a limiting point for 
ijj^'^\u2), ... C 7^1 [as mentioned just after (11), the distances | • |i and | • |;, 
are topologically equivalent so sets of points generate the same limit points 
under both]. 
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Consider any point which is a Hmiting point for Vi, that is, there 

is a sequence of indices ti < t2 < < ■ ■ ■ such that hm„^/'(*")((ii) = u. No 
such u can be a pre-image of d2 or ^3, as for each t, \ A^, the set of 
points with fewer than t legitimate pre-images, is open. Hence, around each 
pre-image of d2 or ^3 there is an open interval of points which have only 
finitely many pre-images and so cannot contain any ip^^\di). 

UueVi, then u = for some m which is unique (otherwise u is 

in a finite orbit and Vi cannot be infinite). Without loss of generality, we 
can assume m <ti and then, since the mapping (^(™) is continuous except 
at '(/'W(di) for n<m, di = (j)^'^\u) =liinnij'^^"~"'\di). 

li u ^Vi, fix a small e > and consider the e-neighborhood Ms{u) of u 
as defined in (8). Assume that e is so small that there are no pre-images of 
d2 or ds in J\fe{u)- Let m be the smallest index such that ^'-'"^((ii) G J\fe{u). 
Then A/'e(^^) contains infinitely many points of Vi and each is a pre-image of 
di with the index of at least m. Now map A^l^^) with tf^"^^-) which sends 
tjj^'^\di) to di and all the points of the neighborhood Ms{u) follow the same 
trajectory. As is contracting here, Lp^^\Me{u)) is an interval within 

A/'e(c^i)- This implies that there are infinitely many points of V\ in J\fe{di). 
Letting e | 0, we obtain that di is a limiting point for Vi. □ 

The following statement immediately follows from Lemma 5.2, properties 
of perfect sets and the fact that Vi is infinite and countable. 

Corollary 5.1. The set Vi is a perfect set (= it is closed and every 
point of it is an accumulation point). Therefore, Vi is uncountable, and 
hence, Vi \ Vi is also uncountable. 

Recall that V is the union of Vi and the set of d2 and ^3 and their finitely 
many pre-images. Then V is the union of Vi and the set of d2 and d^ and 
their pre-images, and is also closed. 

Lemma 5.3. Ifu G Vi, then u does not belong to any finite orbit and no 
trajectory z{t) with z{0) = u can be periodic. 

Proof . If u G Pi , then (u) = di for some n. However, if u belonged 
to a finite orbit, then (p^"''\u) = u for some m and, hence, (p^'^\di) = di, 
contradicting the fact that di has more than m distinct pre-images under 

Now suppose u £ Vi\Vi and u belongs to a finite orbit. As d2, d^ ^ A^ 
for some finite n and is closed, we can choose e > so that \di — > e 
for z = 2, 3. Since Vi C A^, it follows that the neighborhood N£{u) contains 
no pre-images of d2 or 0^3. Let n = n{e) be the smallest index such that 
ip^'^\di) eAfeiu). As 93^") does not split this neighborhood, (A4('w)) C 
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■f^2e{di) and so there is a point of the orbit, n', say, with \u' — di\i < 2e. 
Letting e | 0, we obtain a contradiction, since this orbit contains only m 
distinct points and does not contain di. 

Finahy, consider any trajectory z{t) with z{0) = u and suppose it is pe- 
riodic. By Lemma 4.6(i), z{t) converges onto an orbit wi, Wm and 
we have just shown there exists e > such that Me{wi) CiVi = 0. How- 
ever, z{t) = Lp^*\u) £ Me{wi) for some large t, while simultaneously, by 
Lemma 5.2, ip^^\u) is a limiting point and, hence, an element of Vi which 
is impossible. □ 

Proof of Theorem 3.3. Lemma 5.1 explains how to construct di 
with an infinite sequence of pre-images under (p. Also, by Lemma 5.1, the 
binary sequence for di is not periodic and, hence, by Lemma 5.3, there are 
nonperiodic trajectories {z{t) :t > 0} for the triangle process. □ 

This completes the main task of this section, but it is possible to say 
more about the mixture of periodic and nonperiodic trajectories. We know 
from Lemma 4.6(ii) that all periodic trajectories avoiding decision points are 
stable. It is easy to see that there can he stable periodic trajectories even if 
V is infinite. For example, for the decision points given by (14), the sequence 
in (15) is an orbit with period 3. 

We say that a point u £ ^ is an i?- limit (L-limit resp.) for Vi if there 
is a sequence of points ui, U2, ... in n Pi such that '/r(u.„) | u [vr(u„) t u 
resp.], that is, for an i?-limit, is clockwise of Un for each n. The next 

result strengthens Lemma 5.2. 

Lemma 5.4. Each point uGVi is limiting for Vi on both sides, that is, 
it is both an R-limit and an L-limit for Vi. 

Proof. It suffices to show that di is both an i2-limit and an L-limit, 
since all other points of Vi are ^/^-images of di. 

By Lemma 5.2, di is a limiting point for Vi, but suppose that it is just 
an i?-limit and not an L-limit. Then there is 5 > such that: 

(i) there are no Vi points on the interval (di — 5, di), and 

(ii) there are no pre-images of d2 or d^ in the segment [di ,di + S\. 

Since mapping ip reverses the orientation of intervals, \i u = ip^^\di) G 
and n is even, then u is an i?- limit and not an L-limit, while if n is odd, 
then u is an L-limit and not an i2- limit. 

Fix < (5i < (5. Denote := ^^"H^i);"- = and define 



(16) 



r :=min{n:V'^"''(di) G {di,di + Si]}, 
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so di is the point of Vi H [di,di + 6i] with the smallest index. If r is even, 
then we obtain a contradiction since ip^'^\{di,di]) is not split during these 
mappings and ip^'^\di) = di. Since r is even, (p^^^ preserves orientation and 
is contracting so that (p^^\{di,di]) C (di — 5,di], which contradicts the as- 
sumption that {di — 5, di) r\ Vi = . 

Therefore, in any small interval {di,di + 5i), di has r odd and there will 
be infinitely many with odd n within (di,di + 5i). This implies there will 
also be d]" with even m in (di,di + 6i) since (f^^^ reverses orientation and 
is contracting, so ip^'^\{di , di]) C [di,d|) and all the d" G {di,di] with odd 
n> r have been mapped to d""*" with n — r even. 

Now choose d^" in (di, di -|- 6) with the smallest possible even index m > 2 
and choose odd index t < m such that d* G (di,d"^) with di the closest 
of such points to d™. Such a t > r exists by the arguments following (16) 
above. The interval (d]^,d]"] contains no d" with index less than m so, as 
^pi^) reverses orientation and is contracting, Lp^^\d]^) = d^~* E (di — (5, di), 
contradicting the assumption that (di — 5, di) n "Pi = 0. □ 

Lemma 5.5. Fix a point u and suppose u^V . Then u belongs to the 
segment [Z(u),f(u)] = [l,f] such that each of I and r is either a pre-image of 
d2 or d^, or belongs to Vi and l{u) is the largest of such points and f{u) is 
the smallest of such points: 

Z(ii) = max{7r(s) : Side(s) = Side(ti), 7r(s) < 7r(n), s£V}, 
f(ii) = min{7r(s) : Side(s) = Side(ii), 7r(s)>7r(n), s£V}. 

Moreover, the trajectory started from u can be periodic only if both l{u) and 
r{u) are pre-images of d2 or ds . 

Proof. The existence of l{u) and f{u) follows from the fact that V is 
closed. Also, 7r(/(n)) < 7r(f(it)) since the complement of the set V is open. 

Suppose, for example, that l{u) is not a pre-image of d2 or ds, that is, 
l{u) £ Vi- Since there are no pre-images of the decision points on [/(u),m], 
u and l{u) can follow the same trajectory under mapping ip. On the other 
hand, by Lemma 5.3, no trajectory started from /(u) is periodic, hence, the 
trajectory starting from u is not periodic. □ 

Lemma 5.5 implies that the set of points from which periodic trajectories 
can start — call this the periodic set — consists of finitely many intervals, since 
d2 and ds have only finitely many pre-images. Also, the periodic set has 
measure smaller than the total length of the sides of the triangle, since Vi is 
infinite. At the same time. Lemma 5.4 yields that, for any u^V,ii l{u) G Vi, 
then l{u) ^ 7^1, and the same is true for f{u). 
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Call the set of points u for which one of l{u) or f{u) is in Vi, while the 
other is a pre-image of d2 or ^3, semi-periodic. The endpoint which is in Vi 
will be referred to as an aperiodic endpoint. 

Since there are finitely many pre- images of d2 and ds , the length A > of 
the shortest interval in either in the periodic set or the semi-periodic set is 
properly defined. We say that a point belongs to the aperiodic set if it does 
belong either to the periodic or the semi-periodic set. 

6. Behavior of the stochastic process when "P is finite. We can now show 
how the behavior of the stochastic process H introduced in Section 2 is in- 
fluenced by the behavior of the deterministic triangle process. Recalling (9), 
we consider 

C(n) = A(e(r„)), 

the projection onto Si of H at the random times r^, n = 0, 1, 2, . . . , when the 
server switches from one queue to another (where tq = to; a constant to be 
determined), under the transience conditions in (5), that is, Ai//ij = pj < 1 
for each i and p = J2 Pi > ^■ 

We wish to prove Theorem 3.4, namely, that the trajectories of Cl'^) 
a.s. converge onto one of the stable orbits of the triangle process, under 
the assumption that V, the set of pre- images of the decision points under 
ip, is finite and V CiA^ = for some finite t. 

Recall the assumption that the service time distribution with the server at 
queue i has finite variance af. It is convenient, as with the triangle process, 
to use the transformation in Lemma 4.1, so we can assume that each ^ = 1. 
We now spell out some elementary properties of random walks and Poisson 
processes. 

Preliminaries. Consider a random walk Sn = Y^^=i -^t on Z, where So = 
0, E(Xt) = pj — 1 <0 and Var(Xi) = and Xt> -1. Define the stopping 
times Tc = min{n : Sn = — c} for c = 1, 2, . . . and let Ti^n be independent 
copies of Ti so that Tc is equal in distribution to J2t=i'^i,t- Using the 
standard martingales 5.„ + n{pj — 1) and (5.„ + n{pj — 1))^ — na'^, we find 
that E(ri) = 1/(1 - Pj) and Var(ri) = a^/{l - pjf and, hence, that 

E(r,) = c/(l-p,) 

and 

Var(r,) = caV(l-/>i)'. 

Chebyshev's inequality provides the bound P[|Tc — c/(l — pj)\ > c^^'^] < 
cc7^/c^/^(l — pj)^ = c~^/^(T^/(l — pj)^ on the likely size of deviations from 
the mean. 
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Next consider a homogeneous Poisson process with arrivals at rate p and 
let L = Lq + N{Tc), where Lq is constant and N[Tc) the number of arrivals 
by time T^- Standard calculations give E(L) = Lq + pEi{Tc) and Var(L) = 

Var (Tc) + p'Ei{Tc). Applying these to the process H over the interval Tn+i — 
Tn with the server at node j, we have, for i 7^ j, 



(17) 
and 
(18) 



E(e.(T„+i)ie(T„))=e^(T„) + 



Var(^j(r„+i)|^(r„)) =^j(Tn) 



Using Chebyshev's inequality, we have 



(19) 



<Cj(Tn)" 



While typical deviations on the number of arrivals are of order yS,j{Tn), this 
simple bound for larger deviations will be enough for our purposes. 

To estimate the effect of the projection, we apply Taylor's theorem to the 
function 

, a + ui 
g{ui,U2;a,b) = — ■ , 

b + Ui+U2 

where < a < 6 are constants [compare with equation (10)], which provides 
the formula 

/11U2 - /i2(a + ui) 



g{ui + hi,U2 + h2;a,b) - g{ui,U2;a,b) 



{b + ui + U2 + a{hi + /i2))^ 



where a € (0, 1). Applying this at ui + U2 = u with < u^/'^ for i 
2 — the deviation considered in (19) — we obtain the bound 

\g{ui + hi,U2 + h2;a,b) - g{ui,U2;a,b)\ 

{a + ui +ii2)(|^i| + I/12I) 



(20) 



< 



< 



{b + ui+U2 + a{hi + /i2))^ 
l^il + \h2 



l + 2Ml^ + 0(n-2/3; 



Deviations /i,- 



6 + Ml + M2 V 6 + Ui + M2 

< 2u~^^^ as ti — > 00. 
■ 0{^/u) give a smaller bound, but what we have will suffice. 



TRANSIENT POLLING SYSTEMS 



31 



Proof of Theorem 3.4. We start with the stochastic process Wn = 
Ya=i ^iiTn), n = 0,1,2, ... , the total number of jobs at the queues at switch- 
ing epochs subsequent to tq = tQ, some initial time to be chosen below. 
From (17) and (18), we have 

EiWn+i\arn),mirn)) =j) = ^^^Ci(r„) +e^(r„) +efc(T„) 

Q 

i pj 

where 9 = — 1 > 0. As we are only considering threshold switching 

rules, there is some value > (dependent on the decision points) such that 
if the server switches to queue j at time r„, then ^j(r„) > DWn for each j 
and every n. Hence, with =\ min^ DO /{I — Pj) > 0, this implies that {(1 + 
2v)~'^Wn} is a nonnegative submartingale and so Wn grows exponentially 
in mean. At this point we also introduce u' >0 with {1 + u'Y = 1 + v . 

We are assuming that n A* = eventually so, for any sufficiently small 
eo > 0, there exists no(eo) such that \z — z'\i > 2eo whenever z gV, z' £ A'^° , 
that is, the set A^" is at least 2eQ away from the decision points and their 
pre-images. Also, by Lemma 4.2, we can choose uq large enough that A"'° C 
C(7), the set where (p is contracting. We want to show that any stochastic 
trajectory (in projection) enters A^" and then converges onto a trajectory 
of the triangle process. 

Define a sequence of events Gn, n = 0,1,2, ... , depending on parameters 
Wo, eo by [recall (9)] Go(wo) = {^o > ^^o} H {|C(0) — z\i < Eq for some z G 
and 

Gn = {Wn > woil + n {|C(n) - ^{Cin - l))|i < eoz^'(l + i^')""}- 

We first bound P[G„| Pli"^ Gt]. Choose wo large enough that w^^'^ > 1/v or, 
equivalently, w'^^'^ < vw for all w > wo- As E(iy„|W„_i) > {l + 2v)Wn-i, we 
have from (19) 

nWn < Wo{l + vY\Wn-X > Wo{l + l^)"-^] 

< F[\Wn - E{Wn\Wn-l)\ > l^Wo{l + l^^-^lWn-l > Wo{l + U^-^] 

<r/i/-i/3y,~^/^(l + z^')~^"~^^ 

where ij = m&XijkHpi + /Oj)^o-^ + {pi + Pj){l - Pkf'))/{1 - pkY , a loose but 
adequate bound which shows that Wn grows geometrically quickly with large 
probability. 

Now we deal with deviation of the stochastic triangle process from the 
deterministic one. Prom any y G Mi]„ with A(y) = z £ A^\A^~ with the server 
at node j, we have 

(1 - Pj)Zi+ PiZj _ ( (1 - Pj)zi + PiZj \ 



( 1 - Pj) + V ( 1 - pj) {Zi + Zf^) + {pi + ) Zj 
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and multiplying the fraction hyJ2yi/J2yi to switch to un- normalized values, 
we have 



(21) 



'1 - Pj)Zi + PiZj 



(1 - pj)yi + piVj 



(1 - pj){zi + Zj^) + {pi + pj^)zj (1 - p^){yi + y^) + {pi + ■ 



For the stochastic process H starting from ^(t„_i) = y with J2i Vi > 'Wo{l + 
z^)"~^ and the server at j, the bound (19) implies that, for j, 



Ci{Tn) - Vi + P 



^ 2/3 

>y/ 



C(t„_i) =y 



-1/3 



Now the identity (21) and the estimate (20), when applied at .^(rn-i) = y 
with J2yi > w;o(l + i^)""^ and S,i{Tn) in place of the Ui + /ij, immediately 
imply 

P[|C(n) - mn - l))li < 2w^'/\l + i.')-^""'^le(Tn-i)] 

>l- 277<^/^(l + z.')"^""^^- 

This bound only relies upon Wn-i = J2i ^i{'''n-i) > "^0(1 + i^)""^) an event 
which contains fli"^ Gt- As the process H is Markov and we can choose wq 

-j^ /q 1/3 

large enough that 2i(;q < eoi^V(l + and, as above, t(;o >l/z^, wehave 
established, for such wq, that 



n-l 



Gn\ n Gt 



A very similar argument can be used to show that P[Go] > 1 — rjeo when we 

— 1/3 

choose Wo such that 2noWQ < £o- 

As the stochastic trajectory projects into C{'y) (the region of where 
if is contracting), then, as long as the switching decision is the same at Cit) 
and for every t, we have 



n-l 



C(n) - v^(")(C(0)) = E 9^W(C(n - t)) - (^(*+i)(C(n - t - 1)) 

t=Q 
n-l 



t=Q 



<eoz^'E(l + ^')"*<eo 



on 



t=i 



This inequality says that the trajectory C(t) is never further than eq from 
the triangle process trajectory z{t) started at C(0)- By choice of Go, every 
point of z{t) is more than Eq from any dj, so the switching decisions for (^(t) 



TRANSIENT POLLING SYSTEMS 



33 



and z{t) will always be the same. This shows that on f]^ Gt the projections 
of the trajectories S,{t) converge a.s. onto deterministic trajectories of the 
triangle process which by Theorem 3.1 are periodic. 
Finally, 



it=o 



t=i 



t-i 



Gt\r\Gs 



n-1 

>n(i-^^o^'(i+^')"*) 

t=0 



as n 



oo, 



where p(eo) —^1 as Eq ^ 0. This establishes Theorem 3.4, namely, that the 
stochastic trajectories C(0 converge onto the stable orbits of z, as we can 
choose eo as small as we like. □ 

Remark. The argument above is not delicate enough to decide whether 
trajectories C{t) converge with positive probability onto orbits which are 
stable on one side, for example, that are of even length and contain a decision 
point. 

7. The stochastic process when "P is infinite. The proofs of the following 
results employ the results of Section 5.3. 

Lemma 7.1. Let C(^) denote the state of the stochastic system at time 
n. Fix 5 > 0. Define the event E]\r s by 



i \C{N + k) - ^'^''\C{N))\ <6 for allk>0 such that 
\ Side(C(iV + i)) = Side(¥5«(C(iV))) for i = 0,1, ... ,k 



(22) E]\r^s 

Then there exists a.s. N = N{5,uj) such that E^^s occurs. 



Proof. Analogously to Section 6, one can conclude that there is < 
7 < 1 such that a.s., for all large n. 



(23) 



|C(n)-(/.(C(n-l))|<7". 



Let A'^i = Ni{uj) be the smallest of such n and set N := max(A'^i, min{n : 
Y.Zni' <^})- Then as long as C{N + k) and (/?W(C(^)) go through the 
same sequence of sides of the triangle A^, 

m+k)-^^''HciN))\ 

= |C(iV + k)- ipiCiN + k-l)) + ifiCiN + - 1)) - (^((^('=-i)(C(iV)))| 
<^n+fe + |^(Ar + A;-i)-^(^^-i)(C(Ar))|, 



34 



MACPHEE, MENSHIKOV, POPOV AND VOLKOV 



where we used the fact that is contracting. Now the statement of the 
lemma foUows from induction. □ 

RecaU the definition of the aperiodic set and of A at the end of Section 5.3. 

Lemma 7.2. Define the event E by 

E = {the stochastic system never comes 

closer than A/2 to pre-images of d2 or d^}. 
Then, given that the stochastic system starts in the aperiodic set, P(-E) > 0. 

Proof. First of all, we show that there is a constant K > such that if 
u belongs to the semi-periodic set, then Lp^^\u) belongs to the aperiodic set 
for some k < K. Indeed, the image of an interval / of the semi-periodic set 
must be either another semi-periodic interval, or must lie entirely inside an 
aperiodic interval or a semi-periodic interval. However, if Lp^^^ (I) is in one of 
the finitely many semi-periodic intervals for each k, this would imply that 
the points of / are periodic. This contradicts Lemma 5.5. 

Now let S = A/{2K) and suppose the system starts in the aperiodic 
set. By similar arguments to those of Section 6, with positive probabil- 
ity) Z^^^^o < S, where En is defined in (23). If for some n, is in the 
aperiodic set but (^{n + 1) is not, then u := C(n + 1) cannot be "far" from 
the points of Vi since |e„| < 6. Because the intervals of the periodic set are 
separated from the aperiodic set by the intervals of semi-periodic set, u a.s. 
belongs to a semi-periodic interval, say, / := [l,r], where I G Vi and f is a 
pre- image of d2 or ^3. We claim that, within the next K applications of 93, 
u will be "thrown out" of the image of /, or will be in the aperiodic set or 
some semi-periodic interval not far from its aperiodic endpoint again. 

To show this, suppose ({n + k) lies in ip^^\l) for the first K steps. Then, 
by the choice of K, for some k, it will end up in the aperiodic set, and 
also it will not approach Lp^^\f) (which could possibly be d2 or ^3) closer 
than K x5< A/2. On the other hand, if C{n + k - I) e (p''^~'^\l) and yet 
C,{n + k) ^ Lp^^\l), it means that C,{n + k) is no further from Lp^^\f) than 
£n+k and it lies either in the aperiodic set or in a semi-periodic interval not 
far from its aperiodic endpoint ip^^\f). (Conditioned on Yln=Q kn| there 
is not enough randomness for the stochastic system to exit the interval via 
the endpoint which is a pre-image of ^2 or ^3.) □ 

Lemma 7.3. Conditioned on the event E, periodicity of the stochastic 
system implies that di is a limiting point for the trajectory of the stochastic 
system. 
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Proof. Suppose not. Then there is (5 = 5{uj) > such that the points in 
the (5-neighborhood of di are never hit by the stochastic system. Without 
loss of generahty, suppose 5 < A/2. By Lemma 7.1, there exists an N for 
this 6 such that (22) is fulfilled. Then for any k>0, the distance between 
points (p^''\Ci^)) ^-rid C{N + k) does not exceed 5 as long as they follow the 
same trajectory. On the other hand, they will follow the same trajectory, as 
there will never be the decision point di between them, nor decision points 
d2 or d^, since we are conditioning on E. 

Hence, l{ip^^\C{N))) will be also periodic, following the same path as 
ip^^\(^{N)), yielding contradiction with Lemma 5.3. □ 

Now we are able to finish the proof of Theorem 3.5. 

Proof of Theorem 3.5. First, analogously to the proof of Theo- 
rem 3.4, one can easily show that, with positive probability, the stochastic 
system will converge to one of the finite cycles. So, it remains to show that, 
with positive probability, the stochastic system is not periodic. 

Condition on the event E. Suppose that the stochastic system is periodic 
with period m, that is, for some A'"i, we have 

(24) Side(C(?i')) = Side(C(n + m)) whenever n > Ni. 

For the moment consider only one image (/^((ii) = f2{di) of di. Since f2{di) 
is not periodic by Lemmas 5.4 and 5.3, there is a positive integer Ki = 
Ki (m) > m such that 

(25) Side((/j(-^^i)(di)) / Side(9?(^i-™)((ii)). 

Similarly, for the other possible image of di, fsidi), there is K2 = K2{m) > m 
such that 

(26) Side(99(^2)((ii)) / Side(<^(^2-™)(di)), 

under the assumption that ^p{di) G A3. 

Choose (5 > so small, that each of the one-sided (5-neighborhoods of di 
which map onto sides A2 and A3, respectively, does not intersect with di 
for the first K = maxjETi, i('2} applications of 99, and let 5i<5 he the size 
of the smaller of these neighborhoods after K mappings by ip. 

By Lemma 7.3, the stochastic system will hit the 5i/2 neighborhood of 
di at, say, time A^. Since, in fact, the stochastic system will hit this neigh- 
borhood at arbitrary large times, we can suppose that N > Ni and that 
J2'^=N k"l < "^i/^i where e„ is defined in (23). Also, for definiteness suppose 
that C('^) is on the side of di which maps onto Then the stochastic sys- 
tem will follow the image of di which maps onto A2 for the next K steps, in 
the sense Side((^('=)(C(n))) = Side(9?('')(di)) since 6i/2 + 6i/2 < 61. However, 
recall that K was chosen in such a way that the sequence Side(¥5W(di)), 
k = 1,2, . . . , K , cannot be m-periodic, creating the contradiction between 
(24) and (25). □ 
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